Skip to content

[ENH] Recompress files where dcm2niix failed#803

Open
octomike wants to merge 5 commits into
nipy:masterfrom
MPIB:fix/recompress_uncompressed_nii
Open

[ENH] Recompress files where dcm2niix failed#803
octomike wants to merge 5 commits into
nipy:masterfrom
MPIB:fix/recompress_uncompressed_nii

Conversation

@octomike

Copy link
Copy Markdown
Contributor

As threatened in #802 an alternative approach.

  • try to open and read the nifit to verify it wasn't truncated
  • recompress with gzip
  • fail if we couldn't - as opposed to just emit invalid file names (at least fmriprep fails to handle nii.gz that are not gz at all)

I decided to leverage the fact that heudiconv blindly moves all converted output files to nii.gz, even if they are just nii by renaming the successfully re-compressed image to .nii. This feels ugly, but results in the shortest/least-invasive PR.

@codecov

codecov Bot commented Nov 12, 2024

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.64%. Comparing base (9fc2318) to head (22db16e).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #803      +/-   ##
==========================================
+ Coverage   83.24%   83.64%   +0.40%     
==========================================
  Files          42       42              
  Lines        4404     4488      +84     
==========================================
+ Hits         3666     3754      +88     
+ Misses        738      734       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@octomike octomike force-pushed the fix/recompress_uncompressed_nii branch from 7aff494 to a72d5ec Compare November 12, 2024 23:41
Comment thread heudiconv/convert.py Outdated
"trying to salvage by recompressing ourselves. "
"This might take a while ")
if not recompress_failed(uncompressed):
raise RuntimeError("Error compressing nifti")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why to delay this crash here? this way you are hiding from user information on individual file and corresponding error -- why not just to just call recompress here and let error (if any happens) to bubble up?

I would also just loop here and have recompress to operate on a single file.

FWIW, I think it would be worth adding a test scenario to test this code path. I would have just mock patched the nipype_convert so that I would decompress some output file(s) and thus triggering this code path.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll look into it!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @octomike , have time to finish this one up?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, completely forgot. Will definitely look again

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me just 6 months..

I addressed your suggestions and also added two test cases as you described (disclaimer: those were generated in a claude session - I don't know the policy here, but I'm open to redo those manually). The test cases check that recompress_failed actually recompresses the non-gz nifti returned from a mocked nipype_convert and also assert an error when loading a truncated/corrupt file because in the implementation I validate the uncompressed files are actually okay with nibabel.load().get_fdata().

@octomike octomike force-pushed the fix/recompress_uncompressed_nii branch from a72d5ec to b133a4a Compare February 24, 2026 14:14
@octomike octomike changed the title [WIP] Recompress files where dcm2niix failed [ENH] Recompress files where dcm2niix failed Feb 25, 2026
@octomike octomike force-pushed the fix/recompress_uncompressed_nii branch from f360b96 to 22db16e Compare April 10, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants